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ABSTRACT 

A strategy for cosparing tvo sets of results (one 
based upon early childhood recollections (ECR) and another upon video 
taped (VT) group behavior) fros the Perceptual Characteristics Rating 
Scale vas developed. The null distribution of the sean deviation vas 
estimated by randosly matching an ECR response vector vith a YT 
response vector. To evaluate the degree of relationship of ECR*s and 
VT*s for each subject, a correct pairing vas made. The observed 
values of the lean deviations vere compared to the null distributions 
and the associated empirical probabilities vere converted to chi 
squares to allov summary tests of **no relationship.^ (Author) 
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IMTRODUCTJtN AND OBJECTIVES 



The primary objectives of this paper are threefold: (1) to identify 
various strategies for analyzing dat^i of a specific type vfhich occurs faialy 
often, (2) to compare the statistical properties of these designated stra- 
tegies and (3) to introduce the educational research community to a rather 
simple general strategy for handling certain unusual' data analysis requests. 
The situation in which the data arises is one in which a respondent, often a 
rater, is called upon to rate some kind of observation he has made on a 
unidlmensional scale consisting of a modest number of items. Subsequently, 
he is asked to perform the same task for some other observation which, in some 
way, is associated with the first. The general problem these writers wish 
to address is what recommendation to make to the researcher who wishes to 
determine whether or not the respondents were able to implicitly identify 
the association, i.e., is the fit between the data obtained under the two 
modes better than chance? 

For clarity, the specific situation which lead to the preparation of this 
paper was a request to suggest a strategy for comparing two sets of results from 
the Perceptual Characteristics Rating Scale (PCRS). One set of data was ob- 
tained based on 'behavioral predictions' derived from the subjects' early 
childhood recollections (ECR's); and the other set of data was obtained from 
the observation of video taped behavior (VT's), actual behavioral manifesta- 
tions observed in a group setting. The reseaaxh question involved the predic- 
tive power of ECR's regarding behavior; and, therefore, the primary statisti- 
cal analysis was required to provide evidence of a non-chance fit between 
the two sets of responses made by the same subjects. 
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T'ERSPECTIVES AND THEORETICAL FRi\MEHORK 

In reveiwing the literature on rater agreement, e.g.. Nay lor and Dudycha 
(1967), Naylor and Schenck (1966), Taylor (1968) Taylor, et. al., (1970), little 
research was uncovered which bore directly on the i,Toblm under consideration 
here. (Most of the above studies dealt with psychometric properties of rating 
scales, usually reliability;) Although the cwjonical correlation appears to 
be the classical technique required to attack the problem, no such applications 
were found. Assuming unidiroensionality, the first cannonical r should contain 
the information on the association between the pairs of ratings. This, then, 
appears to be the most reasonable classical technique. Thinking that possi- 
bly alternative strategies might exist for analyzing these data, these researchers 
considered other approaches. 

One would think, for example, that since the problem is one of atiociation, 
some fora of ordinary Pearson r might be appropriate. One migiht, for example, 
compute correlations between the associated response pairs and attempt to evalu- 
ate the magnitude of the computed r's. As is true in Q methodology, the respon- 
ses entering into the computation of the correlations are not independent but, 
there are more serious objections to the use of the Pearson r for this purpose. 
Realizing that there would be N correlations to be evaluated, if N is the num- 
ber of pairs of objects or situations rated, it would seem to be possible to 
compare the distribution of the computed r's with the null distribution. Tliere 
are two serious drawbacks to this approach which invalidate it as a possible 
analytic method. They are (1) since we are thinking of situations in which 
there are relatively few items per scale, the degrees of freedom associated with 
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the nu'l distribution of r would be exceedingly ^Ball but more importantly 
(2) the r distribution would be highly dependent upon the average it^m respon- 
ses. For exainple if the items in a scale differed considerably in mean response 
(regardless of stimulus mode) the responses under the two modes could reflect 
an artificial correlation even though there was no actual association between 
the item respoi^es made by a rater under the two modes. Thus, this approach 
will not be considered further. Although multidimensional scaling might ini- 
tially appear to be useful* it is not appropriate for these data. (See^ e.g., 
Kruskal, 196«*a, 196^b.) 

The final approach to be considered hero will be to conqpute various 
measure of the similarity of or distance between .:he two points (the sets of 
responses under the two modes) in p dimensioniid space where p is the number of 
responses under each mode. The problems with this approach are twofold: the 
researcher (1) must select from among various similarity and distance measxires 
and (2) must obtain the null distribution of the selected distance measure. 
In addition to the correlation, the coraroon measures that come to mind are the 
ordinary Euclidean distance and the Mahalanobl^* distance. The former seems 
quite reasoncible but the validity of the latter is somewhat questionable for 
this situation. The two vector random vau?iables entering into the Mahalano- 
bis distance are assixmed to be independent and have a comnon dispersion matrix. 
The first reqxiireroent is definitely not true although the second might be 
reasonably well satisfied. If the two sample covariance matrices were found 
to be similar possibly some overall estimate would provide useful results. 
For these treasons and others described latsr the Mahalanobis distance was not 
used in this research. Although not generally categorized as a distance 
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measure, and causing pb^dous-mathwiwtical difficulties*' "the average Cabsolute) 
deviation might be considered because olW<*..Aifliplicity^ For this study, then, 
similarity-distance measures were limited to Pearson r (R), absolute avei«age 
deviation (D) and Eudicean distance (E). 

Although the selection of a simUarity-distance measure might cause the 
researcher some difficulty, the problem of how to evaluate the resulting measures 
is an even more troublesome problem. To the knowledge of these writers there 
is no 'reasonably well known' solution to the distribution theory problems 
raised here. It appeared to these researchers that if any of these similarity- 
distance measures were to be used the null distributions would have to be 6b- 
tiined before any evaluation of the subject responses would be possible. 

METHOD AND TECHNIQUES 

Although the ipitial interest to these writers was relative to the speci- 
fic problem of the PCRS data, and a few results from that study wiU be pre- 
sented here, the primary thrust in this paper is a limited methodological study 
to, at least partially, evaluate the usefulness of a non-parametric solution to 
the problem in relation to other possible strategies. The non-parametric ap- 
proach was developed because of the 'distributional problem' mentioned earlier. 
Before going Into the details of the methodological study, the reader is due 
a little more information concerning the original research from which this 
paper derived. (Taylor, 1973) 

A total of subjects were involved in- the study being primarily graduate 
students in education enrolled in "Group Procedures in Guidance" at a southern 
university. The study was conducted during the 1972-73 school year. Twenty 



%e writers are currently looking into other measures of similarity, e.g., 
the sum of cross products and covariance and other measures of distance, e.g., 
norms . 
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.ubject, ««* f.^. >nd 27 The student. fbr th. purposes 

Of instruction, divided into five -class..' of 5. U. 9. 9. snd 13 students. 
These -classes' constituted th. groups which w«« vid«taped. Each tape 
«. approKleately one hov» In length. Each subject «,s «,u.st.d to provid. 
the six earliest instances that he or .he could recaU fro« chUdhoo*. The 
,„hlect sex «s also «ted on the ER fom. T«o novices (other graduate 
student.) «.d t«o experts (inetructor.) we« requested to co^lete a edi- 
fication of CO*. (1969) PCBS based on the ER's. Th. edified KBS contains 
„ it». each of which is rated on a 1-7 scale. The W lte«s are considered 
to provide «a.ure«nt on ftour .«b.cale. as follows: g.n«al paroeptual 
orientation (2 It...). P-ception, of other p«Ple lt«.). perception, of 
..If (5 it«s) and purpose, of behavior (3 it.»s). Eollowing a 'cooling off 
period each of *e four «t.r. was asXed to perfc™ the sa» tasK b...d on th. 
Vt'.. The «te« were .Uowed to view the tape. a. often a. nece...xy to ,: . • . 

accomplish the task. 

Since the .ex of th. subject could be identified under both »od... it 
^ f.lt n.ce..ary to test to determine -ether there were any differences which 
„„ associated w,th the sex of the select. Although the affect would probably 
be slight, a sex difference could cause the agreement aeng th. respons.. 

th. two mode, to he better th«, chance without there being any tn» aaso- 
elation a».g the data. Multivariate ^yis of variance (H«.OVA) was used to 
co^re th. response. f.r^. «.d f,«.l. .ub,.ct. individuaUy.*y r*t.r. Tha 
^t. indicafd that there w.. no .yt^atic variation ..»ci.t.d with th. sex 
of the subject whose behavior or early recollections were b.ing .valuat«l. 

To c«Ty out the 'tert of a.««i.tion' a co,„put.r progr». wa. written 
^ich. fer «ch ratar and seal, c^ination (Wie in all) «nd»ly »tch.d 
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ER and VT responce vectors a total^f._5flCL times. The Pearson r (R) and average 
absolute deviation (D) were coinputed for each random match and an empirical 
distribution was obtained for each. The same statistics were computed, then, 
for each of the «♦? 'correct' matches, i.e., when the same ER and VT vectors 
were matched. The 'taU probabUities ' P; were obtained for each by comparing 
the obtained value to the corresponding empirical 'null distribution' . 
The reader will note that the 'tail' of interest for R includes those values 
near 1.0 while the corresponding 'tails' for D and E are those values near 
0.0. For each of the 16 combinations, then, the quantity 



X^* E -2to P 



was obtained for each of the two statistics. Under the null hypothesis, 
this statistic should be approximately a chi square variable with 9U degrees 
of freedom. Table 1 below contains the results of this analysis for the 
average deviation D. 



Table 1 about here 



Except for subscale U we note the lack of consistency in the data across 
rater. In the original study the chi squares for each subscale were summed 
for 'experts' (raters 1 and 2) and novices (raters 3 and tested sepai^ately 
for significance and compared with an F test. The details of this analysis 
are not presented here because they are not relevant to the primary thrust 
of this paper. Instead, we wiU describe the data which went into the above 
table and look at the results of applying alternative analytic methods. 
The following table presents some correlational data which only partially 



ERIC g 



•7- 



agree with the results above. 



Table 2 about here 



The agreenent between the significant chi squares and the modest average 
correlations between ER and VT is only partial ♦ (fcviously the correlation 
as a measure of similarity addresses different aspects of the data than 
the average absolute deviation. In order to attenf to sunmarize the seeming 
lack of association indicated in Table 2 cannonical correlation analysis 
was performed for each scale and rater* The VT responses were viewed as 
the dependent variables and were regressed onto the corresponding ER*s* 
The results are fo\md in Table 3. 



Table 3 about here 



We note that the cannonical correlation approach identified as significant 
only responses of raters 1 and 2 to subscale ^. Noting the descrepancy 
between the above results and wishing to investigate the relative merits 
of various strategies mentioned etbove^ these researchers launched out on 
a simulation study* 

The parameters for the simulation were selected with an ^e on the 
descriptive data from the present study* Regarding the means and standard 
deviations of the responses, the most extreme values (1 and 7) were seldom 
used by the raters and the standard deviations for the items were generally 
aTOund 1*0 or somewhat less* The question of whether the items differed 
in average response tendency may be addressed by viewing the graphs in 
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Figure 1* 



Figure 1 about here 
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He note some trends which are quite general across raters. For example, 
for subscale 1 the second item generaUy receives a higher mean response 
than the first. Other trends may also be noted but the variation- in 
•Mter effects' is quite large. The data do suggest, howevw, that spurious 
correlations between ER and VT responses probably occurred for some raters. 

Three values for the number of variables under each response node 
were chosen to cover the situations in this sttt«2br and extend upward a bit. 
TTia values selected were three, five and ten variables. Since the data 
tended to support a comnon dimension within the responses under each node, 
and the 'within node' correlations were reasonablj siitilar, it was felt that, 
for simplicity a comnon intercorrelation should be used to relate the 
various responses within a set. The two values chosen for this correlation 
were 0.5 and 0.7 and hereafter this parameter wUl be denoted as RIM. 
The extent to which the responses under the two modes were in agreement was 
considewd next. A correlation of 0 was deemed as necessary to investigate 
the null properties of the procedures. Although an argument could be put 
forth that the two responses to the same item in a scale should be more 
highly related than the responses to two different items, the data did not 
support this notion (actually the associations were so limited, the data 
really gave little infbnnation on this quettion). For simplicity, the 
non-null measures of association between responses under the two modes were 
taken to be 0.3 and 0.5 and this parameter wUl be denoted ao RBET. 
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The question of the validity of these choices and their effect on the 
sijiiilarit3r measures nmst be considered* Since the standard deviations were 
taken as constant (see below) the ^Vithin mode" covariance matrix will exhibit 
compound synnetry. In this sitxiation^ the Hahalanobis distance should be no 
more informative than the Euclidean distance since the weights (elements of 
the inverse of the covariance matrix) would be constant. This fact provided 
yet another reason for exclusion of the Hahalanobis distance function from 
this study. 



Another issue is the use of a constant correlation matrix for the 
relationships between responses under the two modes* This does a dis*- 
service to the Pearson r which addresses the similarity of the two vectors 
on an item by item basis. Under this model, the only way that the correlation 
cannot distribute around zero is if the item means vary consistently for 
the two modes* This is a group effect, however, and not the type of effect 
that we are interested in identijfying. Thus, if the interrelationship 
between the responses under the two modes is ''global" rather than "item 
specific," the R approach should not be expected to identify it. As we 
view the data, we will observe the rather capricio\is behavior of R« It 
was generally felt that it would be unreasonable to expect that RBET 
would be as large as RIN; and , therefore, the following five combinations 
were selected for simulation « 



Combinations of RIN and RBET 



Used in the Simulation 



Simulation 



RIN 
.5 
.5 
.7 
.7 
.7 



RBET 
.0 
.3 
.0 
.3 
.5 



1 
2 
3 

5 
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Thus, two null and three non-null conditions were simulated. Although some 
variation was evident in the data regarding the item means, all were taken to 
be U.O and the item standard deviations for all items were set at 1*0 in reason- 
able accord with the data. Although the data were not checked in this regard, 
a pseudo random normal generator GAUSS, an SSP subroutine, was used to generate 
the data* In keeping with the size of the original study, data sets for SO 
subjects were generated in the following fashion: 

1, The peur^eters of number of variables (NVAR) means, standard devia** 
tions, RIN and RBET, were read in from control cards 

2. The full 2xNVAR correlation matrix was developed and factored using 
a Cholesky factorization algorithm 

3. Standard :;ormal pseudo random vactors were generated and multiplied 
by the factored correlation matrix 

4, Finally, the results were rescaled, translated and rounded off to 
whole numbers in the range l*-7* 

The above procedure was repeated 50 times and the cannonical correlation 
rout in , CANOR of the SSP Subroutine Library, was used to compute the first 
cannonical r and evaluate its significance with an approximate chi square 
procedure described by Cooley and Lohnes (1962). These results were tabulated 
to give power estimates for .05 and .01 percent level tests. There was no 
permutation of the data in this phase of the study. The results will be 
found in thtt sunimary table at the end of this section* 

The principle underlying this section is a rather simple one and repre- 
sents, what these researchers believe to be, a rath^^-? unique analytic strategy 
for the problem at hand. Given the sample of 50 pairs of vectors, X^^ and Xi 
say, the problra is to determine whether they are associated. The null hypothe- 

erJc 



-11 



iof no association suggests that the match between and ^j, the two vectors 
for the ith subject, should be no better on the average than the match between 
?i ^i» The task, then, is to generate the erap^-^ 'null' distri- 

bution by randomly matching an X with a Y, computing the ^-.^ ^-ed measures of 
similarity or distance, and casting these into a distribution. Although for 50 
subjects there are 50x50*250 random matches, a sample of 500 such random matches 
was judged to be sufficient to obtain the empirical distributions. The dietri- 
butions were formed by sorting the generated values and developing percent dis- 
tributions based on the unique values. Attempts to apply a fixed number of 
class intervals proved not to be feasible due co the variability in the 'richness' 
of the distt»ibutions under consideration. (We use the tei^ richness to indi- 
cate the degree of continuity in the distribution v) Some data on this will be 
mentioned later* Afxer the empirical distributions were obtained, the statis- 
tics for the 'correct' matches, i.e., X. and were computed and their per- 
centile ranks in the null distributions were computed. Because of tha discrete 
nature of these distributions, especially for D which does not produce 'rich' 
distributions, it is important to develop the distributions in terms of the 
unique values generated rather then to use class intervals. Percentile ranks 
were calculated in the usual fashion taking the percent of case? below the 
value plus one-*half of the percent of cases at the specific value. This was 
felt to be in rea^^onable agreement with the so called 'significance level* 
being the probability of a result this far into the tail or further ♦ (Of coui^se, 
an argument for a mora conservative approach could be made.) 

A well known method of combining the results of independent experiments 
is to sum the values of -2xln(p£)'frhere the are the tail probabilities 
associated with the independent experiments (Johnson and Johnson » 1959). Each 
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of the quantities in • the sum is appiroximately a chi square random variable 
with 2 degrees of freedom. Although the independence of the various statis- 
tics calculated for the 50 subjects in this study wuld be difficult to jus- 
tify, this approach was used so that the resulting statistics, each based on 
Z-2xln(Pi), were compared to the chi '.square distribution with 100 degrees of 
freedom. 

With only 3 variables, there is little variability in the data and, there- 
fore, few unique values for any of the three statistics calculated. There are 
typically about 12 unique values of D and 2H values of E. The number of unique 
values of R varies froa- 15 to 20 for RIN = .5 but is reduced to about 7 unique 
values when the restrictions concomitant with RIN = .7 are applied to the data. 
Statistics D and E are quite consistent for the cases considered. There are 
approximately 3xhVAR unique values of D and twice as many unique values of E. 
Because of the way the information is processed in obtaining B, it is the • 
•richest' distribution for 5 and 10 variables but its distribution is redu'-ed 
whe^ RIN = .7 relative to when RIN = .5. For example, there were 333 uni-iue 
values of R among the 500 permutation of 10 variables for RIN .5, RBET - 0, 

Unfortunately, in order to fuUy investigate the permutation approach 
under discussion, the simulation experiment described above would have to be 
carried out a number of times in order to gain insight into its properties. 
Although this was done in a limited fashion (50 replications) for the cannonical 
correlation, a similar numb-r of replications for the permutation approach 
was not practical. For each set of parameters, however, two independent 
replications were run and for those cells where the results of the two runs 
were inconsistent, further attempts were made to more clearly identify the 
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properties of the procedure. The following table, then, presents a summary of 
the main analyses of the procedures described above. 



Table ^ about here 



RESULTS 

First of all looking at the resiilts for the chi square test of the first 
cannonical correlation, we observe that in all cases the results for the null 
models are in close agreement with the nominal alpha values of .OS and .01. For 
the three non-null models we- see that the empirical power is lowest for the (.7, 
.3) models, larger for the (.5,. 3) models artd reaches a higher (and quite accep- 
table) level for the (.7,. 5) models. In virtually every case, increasing the 
number of variables reduces the power. The power for alpha of .05 is quite 
unacceptable (less than .5) for all (.7, .3) models and the (.5,. 3) models for 
5 or more variables. For the other four simulations, i.e., the (•7,. 5) models 
and the (.5,. 3) model for 3 variables, the power is in the .60 or better range 
for 5% level tests. 

The permutation results tend to follow the same pCittems although compari- 
sons are rather difficult due to the fact that only two replications are repli- 
cations are reported whereas we have SO replications for the cannonical corre- 
lation analysis. The results r^eported in the table are tail probabilities asso- 
ciated with the accumulated chi square statistic and are reported separately for 
each of the two replications. Therefore, they are not comparable to the powar 
results for the cannonical correlation. The magnitudes of the tail probabili- 
ties for D and E are quite similar but may be quite divergent from the correspon 
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uing value of R for the same data. In no cases were tail probabilities less 
then .1 for D and E for 12 independent replications of null models whereas the 
R probability is essentially 0 for two null 3 variable models. One reason for 
the instability of R for 3 variables appears to be that fact that a large 
number of extreme correlations are generated. For replication #2 for the (.5,0) 
3 variable model nearly **0 percent of the permuted R's were -1.0. Tmus the 
17 borrect* matches which produced a -l.Og the most unsatisfactory value from 
a standpoint of positive association, were assigned a tail probability of .80. 
This is simply the largest value possible for the permution distribution gen 
erated. For tiie saioe data, the permutation distribution contained only 9% 
of R values of .0 so that the correct matches yielding this R value, received 
a tail probability of about .045. The average tail probability assigned to the 
50 correct matches was, therefore, .42 and the resulting chl s.quare highly sig- 
nificant • 

The R statistic was capable of correctly identifying a non-null associa- 
tion for the (.5,. 3) 3 variable model. The only other non-null model for 
which R was significant was one of the two replications of the 5 variable, 
(.7,*5) model. Thus, the R statistic does not appear to have the desi^d 
properties. 

Looking at non-null models we observe that D and E are quite similar in 
performsnce with E having the smaller of the tow probabilities about 80% of 
the time (11 of 13 where any difference exists). The E procedure provided 
consistent significant ( .05 level) results for three of the nine non-nuU 
models and the D statistic performed almost as well. For each of the three 10 
variable non-null models and the (.5,. 3) 5 variable slmixlation E was signi- 
ficant on one of the two replications and D was generally also significant or 
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close. The cannonical r power at the 5% level for these four situations were 
estimated to be .22, .20, .68 and .12 so that the four significant replications 
in eight suggests the possibility of some improved performance of the 
permutation approach over the cannonical r for a large number of variables 
ixxpon closer scrutinity, they appear to be about the same) . The permutation 
approach does not appear to identify the association very well when the 
association is strong (.7,. 5 and .7,. 3) and there are only 3 variables. This 
most likely has to do with the 'richness' issue since these two cells exhibit 
the minimum number of unique values (about 7,12 and 23 for R,D and E respec- 
tively) . Three extra runs were made for the four cells in which the results from 
the two replications for E were inconsistent. The three additional runs for 
the 5 variable (.5,. 3) cell produced one significant (.05) result for D and 
E so that 2 of 5 or «+0% of the replications for E produced significance. This 
is in good agreement with the power estimate of .U2 for the cannonical r 
approach. Additional runs for the 10 variable (.5,. 3) model produced no 
further significant results and so the estimate for that cell is .20 based on 
five runs. (The cannonical correlation approach yielded a power estimate of 
.22 .) 

For the 10 variable ( .7,.3) model none of the three additional runs 
were significant (although all of the P's were less than .20) so, once again, 
the estimate is .20 based on five replications, the same as the cannonical esti- 
mate. For the 10 variable (.7,. 5) model, two of the three additional runs were 
significant for both D and E yielding a power estimate of .60 based on five 
replications. This compares quite closely with the estimate based on 
cannonical r of .68. Thus, although of a very limited nature, these results 
are in relatively good agreement with those of the cannonical r. 
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In order to demonstrate that the power of the permutation procedure based 
on D and E, both of which appeared to have reasonable properties, was related to 
the size of the original sanple, two saraples of size 100 were generated for 
each of the three non-null (.T^.S) models. In each case both D and E produced 

/ 

restats significant at the ,01 level for each replication. Thus, although the 
power for samples of size 50 su3?ely differs somewhat from that for the cannonical 
r, the permutation procedure (along with cannonical r, sur»ely) appears to have 
reasonable power properties as n increases. In general E appears to be somewhat 
iDore powerful than D. 

SUMMARY 

Although in the specific situation described here a classical procedure 
for cnalyzing the data existed, the non parametric procedure outlined here 
appears to provide results that compete reasonably well with the cannonical r. 
One of our primary reasons for presenting these data, however, is to illustrate 
the relative simplicity of the technique. It is rather likely that most statis- 
tical consultants are requested, from time to time, to provide recommendations 
for data analyses where no such classiced method is available. For these 
situations it may be possible that a procedxire along the lines of the one 
presented here could be used. The progranming is quite simple requiring a 
uniform random number generator to select random Indices, a subroutine to 
compute the statistics desired and possibly another to form the «npirical 
frequency distributionCs) . The same subroutine can then be used to compute 
the statistics for the correct matches. Then th<^ir percentile ranks need to 
be calculated, transformed and accumulated. We feel that the procedure has 
sufficient merit to be included, possibly toward the bottom, but certainly 
in your 'bag of tricks ' . 
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TABLE 1 



CHI SQUARES COMPUTED FROM THE PROBABILITIES 
OBTAINED BY COMPARING THE DEVIATIONS BETWEEN 
TWO SETS OF RESPONSES ON THE PERCEPTUAL CHARACTERISTICS 
RATING SCALE TO THE COMPUTER-GENERATED 
"RANDOM-MATCHED" DISTRIBUTIW 



Subscales 


! Perceptual 
Characteristics 
Rating Scale 
Categories 


Chi Squares 


Rater 1 








OX 


1 


General 
Perceptxial 
Orientation 
(2 items) 


113.29 


173.94** 


197.65** 


86.88 


94 


2 


Perceptions 
of Other 
People 
items) 


146.97** 


62.58 


96.69 


115.79 


94 


3 


Perceptions 
of 
Self 
(5 items) 


96 /i9 


128. 54* 


102. m 


92.26 


94 




Purposes 

of 
Behavior 
(3 items) 


131.98* 


163.98** 


138.06** 


135.91** 


94 



* P<.05 
** P<.01 
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TABLE 2 



AVERAGE INTER-ITEH CORREUTIONS WITHIN AND BETWEEN HODES 
OF PRESENTATION BY RATER AMD SUBSCALE OF PCRS*" 





. Average Within ER 


. Average Within VT 


Average Between ER S VT 


Rater 
Siibscale 


1 


2 


3 


4 


1 


2 


3 


4 


1 


2 3 4 


1 (2 itenrn) 


.30 


.42 


.47 - 


.16 


.23 


.23 


.55 


.25 


.18 


.06 -.05 -.03 


2 (4 items) 


.77 


.21 


.72 


.51 


.70 


.23 


.53 


.66 


.04 


.03 -.07 -.15 


3 (S items) 


.64 


.47 


.50 


.18 


.72 


.50 


.47 


.56 


.04 


.06 -.05 -.08 


4 (3 items) 


.57 


.32 


.36 


.24 


.39 


.28 


.21 


.51 


.19 


.13 .05 -.11 



* Arithmetic averages were felt to be sufficient for the purposes of 
this presentation although, in gener&l, averages based on Fisher's 
z transformation are preferred. 



TABLE 3 

HRST CANNONICAL CORRELATIONS AND SIGNIFICANCE 
BY RATER AND SUBSCALE OF PCRS 



• 






Rater 






Subseale 




1 


2 


3 


4 




CANR 


.30 


.29 


.16 


.25 


1 (2 items) 




5.0 


^ 4.6 


1.2 


, 3.0 




Prob 


.29 


.34 


.88 


.56 




CANR 


.45 


.46 




137 


2 (4 items) 


(16df) 


14.0 


16.8 


16.3 


11.5 




Prob 


.60 


.40 


,H3 


.78 




CANR 


.60 


, .4^ 


.45 


.47 


3 (5 items) 


(25df) 


34.1 


25.6 


15.5 


19.8 




Prob 


.10 


.43 


.93 


.76 




CANR 


.55 


.49 


.40 


.35 


4 (3 items) 


(9df) 


19.4 


17.2 


12.7 


9.8 




Prob 


.02 


.04 


.17 


.36 
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TABLE »♦ 



Resulta-OiE-Statistical Signif ic^pce- for Four Apgipaches to Analyzing Data for 
Sampl*s-<xf3Q_5i4>5Wt&-JF«r'T'^^ Data Models 



R 
I 


B 
B 


Nuitjer of Variables 


N 


E 






■ r ■ 








! 










T 


CAS. 


+500 


Permutations 


CAN 5f.O Permutations 


CAN 


500 f 


'ermutations 






pink 


R 




E 


r 


R 


D 


E 


r 


R 


D 1 




.5 


.0 


08 
(05) 


U3 




93 


06 
(05) 


US 


87 


75 


06 
(05) 


29 


21 


26 






02 


00 


65 


61 


00 
^01) 


25 


22 


22 


02 
(01) 


29 


15 


16 


.5 


.3 


(05) 


00 


02 


02 


U2 
(05) 


30 


50 


3U 


22 
(05) 


3U 


01 


n't 
01 






20 
-(01) 


02 


OU 


OU 


22 

:oi) 


36 


06 


05 


12 
(01) 


78 


41 


36 


.7 


.0 


08 
(05) 


U7 


72 


63 


OU 
(05) 


62 


23 


18 


06 
(05) 


38 


95 


96 






00 
(01) 


00 


77 


79 


02 
(01) 


U9 


89 


90 


00 
(01) 


81 


10 


10 


.7 


.3 


38 
(05) 


13 


U6 


UO 


28 
(05) 


25 


06 


05 


20 
(05) 


71 


71 


6U 






12 
(01) 


19 


51 


UU- 


12 
(01) 


50 


06 


03 


06 
(01) 


32 


00 


02 


.7 


.5 


9U 
(05) 




25 


18 


88 

(05) 


03 


00 


00 


68 
(05) 


6^ 


03 


03 






80 
(01) . 


75 


06 


09 


66 
(01) 


93 


07 


03 


H2 
(01) 


97 


25 


lU 



* Decimal point omitted in body of table. 

** Results for cannonical correlation coefficient; values feporte<? 
are numbers of significant results using a = .05 and a = .01 for 
the Chi square test of the first cannonical coi^relation. 

+ Results for two replication, each based on 500 permutation of the 
raw data for 50 s\ibjects. 
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FIGURE 1 

SUBSCALE MEAN PROFILE BY RATER AND MODE OF PRESENTATION OF PCRS 
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the nu'l distribution of r would be exceedingly ^»all but more in5)ortantly 
(2) the r distribution would be highly dependent upon the average it^m respon- 
ses. For exan5>le if the items in a scale differed considerably in mean response 
(regardJ#s8 of stimulus mode) the responses under the two modes could reflect 
an artificial correlation even though there was no actual association between 
the item responses made by a rater under the two modes. Thus, this approach 
will not be considered further. Although multidimensional scaling might ini- 
tially appear to be useful, it is not appropriate for these data. (See, e.g., 
Kruskal, 196ila, 196(4b.) 

The final approach to be considered here will be to compute varioiis 
measure of the similarity of or distance betwee:; rhe two points (the sets of 
responses under the two modes) in p dimension^id space where p is the number of 
responses under each mode. The problems with this approach are twofold: the 
researcher (1) must select from among various similarity and distance measxires 
and (2) must obtain the null distribution of the selected distance measure. 
In addition to the correlation, the common measures that come to mind are the 
ox^inary Euclidean distance and the Mahalanobin ^ distance • The former seems 
quite z^^asoncible but the validity of the latter is somewhat questionable for 
this situation. The two vector random variables entering into the Mahalano- 
bis distance are assumed to be independent and have a common dispersion matrix. 
The first requirement is definitely not true although the second might be 
reasonably well satisfied. If the two sample cover iance matrices were found 
to be similar possibly some overall estimate wotild provide useful results # 
For these reasons and others described latsr the Mahalanobis distance was not 
used in this research. Although not generally categorized as a distance 
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